Video aspect ratio manipulation

ABSTRACT

According to the invention, a video distribution system is disclosed. The video distribution system distributes a video program in a first aspect ratio and includes information to convert the first aspect ratio to a second aspect ratio. Included in the video distribution system are a distribution point and a video converter that is located remotely from the distribution point. The video converter receives the information and the video program from the distribution point. The video converter uses the information to guide conversion between the first aspect ratio and the second aspect ratio.

[0001] This application claims the benefit of and is a conversion ofU.S. Provisional Application Ser. No. 60/455,009 filed on Mar. 14, 2003and U.S. Provisional Application Ser. No. 60/476,503 filed on Jun. 6,2003, both of which are incorporated by reference in their entirety.

BACKGROUND OF THE INVENTION

[0002] This invention relates in general to video systems and, morespecifically, to aspect ratio manipulation of video images.

[0003] From its standardization in the 1950's, television displays havetraditionally used a 4:3 aspect ratio, displaying an image that is fourunits wide and three units high. A very large quantity of televisionprograms were produced in the 4:3 aspect ratio. In 1996, the FCC adoptednew display formats for digital television and chose a 16:9 aspect ratiodisplay for high definition television (HDTV). Beginning in 2002, sometelevision production began shifting from the traditional 4:3 aspectratio images to 16:9 images in an attempt to “future proof” the productbecause a 16:9 image may be processed for display on a legacy 4:3display by cutting off the sides in a process called “pan and scan.”Using pan and scan, the center part of the wider aspect ratio image iscropped to produce a 4:3 aspect ratio image. Another conventionaltechnique is the “bob and crop” and is used to convert aspect ratios.

[0004] In FIGS. 1 and 2, a conventional 16:9 display 100 and a 16:9image 200 are respectively shown. In the image depictions of thefigures, hatched circle images are used accentuate any distortion to theimage. FIGS. 3 and 4 respectively show a conventional 4:3 display 300and a conventional 4:3 image 400. Today, conventional televisions areavailable in either format, but other aspect ratios are known. Forexample, movie theatres may display even wider images.

[0005] The pan and scan process uses trained technicians to manuallycrop a center part of the image with dynamic adjustments of a 4:3aperture left or right to capture most of the content of the 16:9 image200. In other words, the edges of the 16:9 image 200 are cut off to fillthe 4:3 display 300. In a similar process, a 16:9 aperture pans andscans up and down through a 4:3 image 400, but cuts off horizontalslices of the 4:3 image 400 below and above the 16:9 aperture.

[0006] During the transition to digital television there will be a timewhen broadcasters will transmit signals with both 4:3 and 16:9 images400, 200. Various consumers will have both 4:3 and 16:9 television sets300, 100 that respectively display both 4:3 and 16:9 images 400, 200.The optimum viewer experience will exist when the display aspect ratioexactly matches the image aspect ratio, for example a 4:3 display 300showing a 4:3 image 300 or a 16:9 display 100 showing a 16:9 image 200.In both these cases, the image fills the display and the image ispresented with all the available horizontal and vertical resolutionpossible in the display.

[0007] Often the image aspect ratio of the image doesn't match theaspect ratio of their TV display. This will be an ongoing problembecause of the large amount of television programs produced in 4:3aspect ratio. On the other hand, theatrical movies are produced inwidescreen format with an aspect ratio of 1.85 or wider, which causesaspect ratio problems also.

[0008] There are two conventional ways to present a 16:9 image 200 on a4:3 display 300, as shown in FIGS. 5 and 6. FIG. 5 uses letterbox format500 to match the width of the 4:3 display with the width of the 16:9image 200, producing black bands 508 at the top and bottom of the 4:3display 300. The 16:9 image height is reduced to 75% of the 4:3 screenheight, and the overall displayed image 200 is reduced to 75% of thedisplay area. Because of the fixed vertical scanning structure (physicalpixels on LCD and DLP sets, or scan lines on CRT-based displays),available vertical resolution is effectively reduced 25% by theletterbox display 500.

[0009] Another problem with the letterbox display 500 is consumerdissatisfaction. When the image doesn't fill the screen, most consumersfeel like they are being short-changed, that they are not getting thefull value from their television sets. And the black bands 508 aredistracting from the viewing experience. Yet another problem with theletterbox display 500 is “burn in” on some types of displays. Since somedisplay elements (or phosphors) are not used in the letterbox display500, the display elements age differently, and can result in the blackbands 508 being “burned in” to the display 500, showing a persistentband even when not using letterbox format.

[0010] The second conventional approach to fill the 4:3 display with a16:9 image is to stretch the image vertically as shown in FIG. 6. Thisapproach would use the entire vertical scanning capability of the 4:3display 300 to deliver the maximum available vertical resolution. Inpractice, this approach produces unacceptable geometric distortion whichvertically elongates the image 604.

[0011] In conventional systems, 4:3 images 400 are commonly shown on16:9 displays 100 using a pillar box presentation 700, with blackvertical bars 708 on the left and right of the image 400. Using thisapproach that is shown in FIG. 7, the 4:3 image 400 only uses 75% of theavailable 16:9 screen width. In other words, the 4:3 image 400 uses only75% of the 16:9 display area. The horizontal resolution is reduced by25% since every scan line of the image 400 does not utilize all theavailable pixels of a scan line of the 16:9 display 100. The previousproblems with consumer satisfaction and bum-in discussed above alsoapply to the pillar box display 700. In this case, the vertical pillarboxes 708 will be burned in.

[0012] Another approach 800, shown in FIG. 8, is to stretch the 4:3image 804 to match the width of the 16:9 display 100. The resultinggeometric distortion may be tolerable for some kinds of viewing, but thegeometric distortion is always present nonetheless.

[0013] In conventional systems, some variations on the horizontalstretch apply a more pronounced stretch at the left and right edges ofthe image, and apply less stretch in the center of the picture. Thisnon-linear type of stretch leaves the geometry in the center of theimage relatively unchanged but at the expense of more radical geometricdistortions at the left and right edges.

BRIEF DESCRIPTION OF THE DRAWINGS

[0014] The present invention is described in conjunction with theappended figures:

[0015]FIG. 1 is a diagram of a conventional 16:9 aspect ratio display;

[0016]FIG. 2 is a diagram of a conventional 16:9 aspect ratio image;

[0017]FIG. 3 is a diagram of a conventional 4:3 aspect ratio display;

[0018]FIG. 4 is a diagram of a conventional 4:3 aspect ratio image;

[0019]FIG. 5 is a diagram of a 4:3 aspect ratio display that shows a16:9 aspect ratio image in a conventional letterbox format;

[0020]FIG. 6 is a diagram of a 4:3 aspect ratio display that shows a16:9 aspect ratio image using a conventional vertical stretch;

[0021]FIG. 7 is a diagram of a 16:9 aspect ratio display that shows a4:3 aspect ratio image in a conventional pillar box format;

[0022]FIG. 8 is a diagram of a 16:9 aspect ratio display that shows a4:3 aspect ratio image using a conventional horizontal stretch;

[0023]FIG. 9 is a block diagram of an embodiment of a video distributionsystem;

[0024]FIG. 10 is a diagram of an embodiment of an 16:9 aspect ratioimage with an 4:3 aspect ratio aperture overlaid;

[0025]FIG. 11 is a diagram of an embodiment of an 4:3 aspect ratio imagewith to an 16:9 aspect ratio aperture overlaid;

[0026]FIG. 12 is a diagram of yet another embodiment of an 16:9 aspectratio image with an 4:3 aspect ratio aperture moving with two degrees offreedom;

[0027]FIG. 13 is a diagram of an embodiment of an 4:3 aspect ratio imagewith a first and a second apertures overlaid;

[0028]FIG. 14 is a diagram of an embodiment of a 16:9 aspect ratio imagethat has portions selected for conversion to a 4:3 aspect ratio image;

[0029]FIG. 15 is a diagram of an embodiment of a 4:3 aspect ratio imageproduced from portions selected in FIG. 14;

[0030]FIG. 16 is a flow diagram of an embodiment of a process forprocessing a video image for distribution to displays having differentaspect ratios;

[0031]FIG. 17 is a flow diagram of an embodiment of a process forprocessing a video image for showing on a display having a differentaspect ratio;

[0032]FIG. 18A is a block diagram of an embodiment of a direct broadcastsatellite (DBS) system;

[0033]FIG. 18B is a block diagram of an embodiment of a cable system;

[0034]FIG. 19A is a block diagram of an embodiment of a video processingsystem for the DBS system;

[0035]FIG. 19B is a block diagram of another embodiment of the videoprocessing system for the cable system;

[0036]FIG. 20A is a block diagram of an embodiment of a set top box; and

[0037]FIG. 20B is a block diagram of another embodiment of the set topbox with program storage capability.

[0038] In the appended figures, similar components and/or features mayhave the same reference label. Further, various components of the sametype may be distinguished by following the reference label by a dash anda second label that distinguishes among the similar components. If onlythe first reference label is used in the specification, the descriptionis applicable to any one of the similar components having the same firstreference label irrespective of the second reference label.

DETAILED DESCRIPTION OF THE PREFERRED EMBODIMENT

[0039] The ensuing description provides preferred exemplaryembodiment(s) only, and is not intended to limit the scope,applicability or configuration of the invention. Rather, the ensuingdescription of the preferred exemplary embodiment(s) will provide thoseskilled in the art with an enabling description for implementing apreferred exemplary embodiment of the invention. It being understoodthat various changes may be made in the function and arrangement ofelements without departing from the spirit and scope of the invention asset forth in the appended claims.

[0040] In one embodiment, the present invention provides a videodistribution system. The video distribution system distributes a videoprogram in a first aspect ratio and includes information to convert thefirst aspect ratio to a second aspect ratio. Included in the videodistribution system are a distribution point and a video converterremote the distribution point. The video converter receives theinformation and the video program from the distribution point. The videoconverter uses the information to guide conversion between the firstaspect ratio and the second aspect ratio.

[0041] Referring first to FIG. 9, a block diagram of an embodiment of avideo distribution system 900 is shown. This system 900 streams videoprograms to a number of users in a broadcast or single cast topology.The video programs are received from any number of sources, including, asatellite receiver 916, a wide area network (WAN) 910, a wirelessantenna 912, and/or a media reader 922. The video programs may be storedin a video store 904 for later playback and/or streamed to thetransmission system 908 when received. Programs on the video store 904may be played in any on demand format, including, VOD, SVOD, pay perview, NVOD, remote PVR, etc.

[0042] The transmission system 908 could have many differentconfigurations. For example, communication may be one way orbi-directional. In one embodiment, the transmission system 908 includesa hybrid fiber/coaxial transmission media and may include neighborhoodnodes. Another embodiment uses a satellite link in the transmissionsystem 908 to communicate video programs. Other embodiments may usevDSL, wireless television (e.g., wireless VOD, microwave line of sight,non-line of sight, etc.), terrestrial broadcast, and/or Internetdistribution (broadband modem, DSL, cable modem, wireless data,satellite data, etc.) in the transmission system 908.

[0043] Video images distributed by the system 900 may include videoaspect ratio conversion (ARC) data 928 to indicate how to convert fromone aspect ratio to another. Where no ARC data 928 is available, thevideo conversion may use some conventional technique. For any programsthat are played in the future, the conversion parameters are stored in avideo ARC database 928. The video ARC data may be embedded as metadatain a video stream or sent separately from the video stream. In someembodiments, the headend can automatically generate the video ARC datawhere there is none.

[0044] A video converter 920 processes a received video image having afirst aspect ratio to produce another video image having a second aspectratio. In some cases, the first and second aspect ratios are the same,but only a subset of the received video image is used to produce theother video image. Some video converters 920 know the aspect ratiopreferred by a video display 932 coupled to the video converter 920.Other video converters 920 produce a number of output aspect ratios fromthe received video image and the user can connect to any of them.

[0045] The video converter 920 may sense when an output cable isattached to a given port with a defined aspect ratio before producing avideo image in that defined aspect ratio. Other embodiments, allow theuser to configure the video converter for the aspect ratio(s) that willbe used. The video converter could be located in the user location(e.g., within the set top box, video receiver, television, video modem,computer, and/or other equipment) or could be located remote to the userlocation (e.g., in a neighborhood node, the headend, a satellite, asatellite dish, an external video receiver, and/or other equipment).Some video equipment already has the ability to process video to changethe aspect ratio and a simple software or firmware upgrade may allowremote manipulation of this process.

[0046] In various embodiments, the video display could be a movietheatre projector, a home theatre projector, a flat panel display, a CRTdisplay, a television, a computer display, 3D display, a personal videoplayer, a DVD player, a personal digital assistant (PDA), a wirelesstelephone, etc. Any aspect ratio conversion could be supported from asingle video stream. The video display could be wired to the videoconverter or be wireless. Home theaters with curtains or masking tomatch the aspect ratio could be automatically activated according to thechosen aspect ratio for display.

[0047] A user of the video converter 920 may be able to customize howthe aspect ratio is converted for one or more programs. In the simplestform, the user could override the video ARC data to use a letterbox orpillar box format. Other customizations include disabling non-linearscaling, aperture rotation, etc. This could be done for an individualprogram or for all programs. There may be a select number of presetprofiles available to the user and some could be customized. Forexample, the automatic selection would just use the ARC data, butanother setting would take the 16:9 aspect ratio and letterbox it.

[0048] In some embodiments, a PVR may record the received video imagebefore conversion. The video ARC data would also be stored to allowlater conversion according to the video display chosen for playback. ThePVR may serve any number of displays having different aspect ratios andthe conversion could be done when playback is selected. Some embodimentsmay perform the conversion before storing on the PVR.

[0049] With reference to FIG. 10, a diagram of an embodiment 1000 of the16:9 aspect ratio image 200 is shown with an 4:3 aspect ratio apertureor cutout 1012 overlaid thereon. The video image 200 is received in a16:9 aspect ratio, but then processed by the video converter 920 to onlydisplay a portion corresponding to the 4:3 aperture 1012. A horizontalposition 1008 is an offset of the right edge of the 4:3 aperture from avertical centerline 1004 of the 16:9 image. This horizontal position1008 varies to capture the best positioning of the 4:3 aperture. Changesin the horizontal position 1008 are communicated to the video converter920.

[0050] The position of the 4:3 aspect ratio aperture 1012 could bechosen by the director, a technician or desired automatically. In oneembodiment, a computer compares two versions of the program havingdifferent aspect ratios to determine how the movement of the aperturewas chosen. Often programs are available on DVD or to content providersin two alternative aspect ratios. Rather than transporting both formatsto each user, the content provider only provides one aspect ratio, butincludes video ARC data so the video converter 920 can more effectivelyproduce the aspect ratio desired by the user.

[0051] Other embodiments might also specify a vertical position for the4:3 aperture 1012. Where the aperture 1012 doesn't match the resolutionof the output aspect ratio, an up-conversion or down-conversion isperformed. Some embodiments may provide for a non-linear scaling suchthat one portion of the 4:3 aperture 1012 is scaled more than anotherportion. When it is known that the ARC data will indicate scaling,graphics, text and images can be chosen that are less prone to show thescaling. In various embodiments, a video signal of a given aspect ratiomay have any number of resolutions. For example, the video signal may bea 4:3 aspect ratio, but have enough resolution to support taking a 16:9cutout without any loss of resolution.

[0052] Referring next to FIG. 11, a diagram of an embodiment 1100 of the4:3 aspect ratio image 400 with an 16:9 aspect ratio aperture 1112 isshown. A vertical position 1108 indicates the variance of the bottom ofthe 16:9 aperture 1112 from a horizontal centerline 1104. Although thisembodiment only has one degree of freedom for the aperture, otherembodiments could also allow specifying the horizontal position,scaling, angular rotation, any mirroring, image correction, aperturesize, scaling algorithm, and/or other video processing to be performedby the video converter 920.

[0053] In one embodiment, a single 16:9 aspect video image has multiplevideo streams. For example, different camera angles for a sports programmay be included. The user may be able to specify to the video converterwhich video stream to display by moving an aperture through the receivedvideo image. If the user did not change the selection, the video ARCdata would be used to move the aperture between the various videostreams.

[0054] With reference to FIG. 12, a diagram of yet another embodiment1200 of the 16:9 aspect ratio image 200 with the 4:3 aspect ratioaperture 1012 moving with two degrees of freedom. The horizontal andvertical positions 1008, 1108 indicate an offset of the 4:3 aperture1012 from a center of the 16:9 image 200. The 4:3 aperture could be ofany size and later scaled by the video converter 920 for the resolutionof the target video display 932.

[0055] Referring next to FIG. 13, a diagram showing an embodiment 1300of the 4:3 aspect ratio image 400 with a first aperture and a secondaperture 1308-1, 1308-2 overlaid. The two apertures 1308 are separatelyspecified and manipulated to produce a 16:9 image that is presented onthe video display 932. In this example, the first aperture 1308-1 istaller than the second aperture 1308-2. A first vertical position 1108-1is specified for the bottom edge of the first aperture 1308-1 and asecond vertical position 1108-2 is specified for the bottom edge of thesecond aperture 1308-2.

[0056] To reformulate the presented video image, the first and secondapertures 1308 are processed and stitched together. In this example, thereceived 4:3 video image 400 is for a news channels which uses a talkinghead and graphics in the first aperture 1308-1 and a barker in thesecond aperture 1308-2 at the bottom of the screen. The first aperture1308-1 is selected such that any important portions of the talking headand graphics are not unusually cropped. The height of the first aperture1308 may be only slightly compressed in height during the conversion.The barker in the second aperture 1308-2 can be more aggressivelycompressed in height during the conversion as the resolution of thedisplay video image will still allow any text to be readable. In thisway, most of the substance of the 4:3 image 400 can be presentedfull-screen on the video display 932 without resort to pillar boxing.

[0057]FIGS. 14 and 15 show an embodiment 1500 of a 16:9 video image 200in FIG. 14 that is converted to a 4:3 display image 1508 in FIG. 15.Three apertures 1308 are selected from the 16:9 image 200 for presentingin on the video display 932 as a 4:3 video image 1508. A first aperture1308-1 maps with the same resolution to a first portion 1504-1 of the4:3 video image 1508. A second aperture 1308-2 is rotated ninety degreesand scaled-up to occupy a second portion 1504-2 of the 4:3 image 1508. Athird aperture 1308-3 is down-scaled to occupy the a third portion1504-3 of the 4:3 image 1508. The video ARC data may change or besuspended during commercial breaks or scene changes.

[0058] Some embodiments could indicate the mapping from aperture 1308 toportion 1504 on the production equipment such that the ARC data could betested when the video is captured. For example, the mapping could bedefined in the camera and two screens could show the different aspectratios.

[0059] With reference to FIG. 16, a flow diagram 1600 of an embodimentof a process for processing a video image 200, 400 for distribution todisplays 932 having different aspect ratios is shown. This processingcould be performed in the headend, by a content provider or some partyrelated to the owner of the program. Where there are already multipleversions of a program having different aspect ratios, the video ARC datacould be automatically determined. A technician may have to check andclean-up the automated determination.

[0060] The depicted portion of the process 1600 begins in step 1604,where the source video is obtained in a predetermined aspect ratio. Inan iterative manner, cutouts or apertures 1308 are chosen from thesource video and assigned specified manipulations and mapped to theoutput video until all desired portions 1504 the output video areoccupied. Each output aspect ratio has its video ARC data determined insuccession.

[0061] In step 1608, a location of the cutout or aperture 1308 isdetermined. The manipulations suggested to the video converter 920 arespecified in steps 1612, 1616 and 1620. Horizontal and vertical scalingis determined in step 1612. This may increase or decrease the pixels forthe aperture 1308 such that the portion 1504 is occupied on thedisplayed image. The technician may specify non-linear scaling such thatalong a horizontal or vertical axis, the scaling changes. A histogrammay be used to allow specifying the variance in scaling.

[0062] The aperture 1308 may have a rotation or mirror effect specifiedin step 1616. Placement of the processed cutout is determined in step1620. The specification of apertures 1308 and their processing continuesby looping back to step 1608. When all the video ARC data is specifiedfor a given scene, the source video is advanced to the next scene withstep 1626. Specification of the video ARC data continues iteratively forall the scenes of the source video. The video ARC data indicates thetiming for changes in the processing throughout the program. Whencomplete, step 1628 would loop back to step 1608 where additional aspectratios to play the video should be specified. Once all aspect ratios areaddressed, the video and the ARC data are distributed to the video store904 or simply played in a linear schedule.

[0063] Some embodiments may leave some portions 1504 of the output videoimage blank and fill that space with any color background.Alternatively, advertising messages could be placed in the unusedportion 1504. This could include filling the unused portions 508, 708when doing a standard letterbox or pillar box presentation. Statusinformation, cross-channel promotions and/or advertising could be placedin the unused portions 508, 708, 1504. The information to populate theunused portions could be sent in real time from the headend or sentearlier and stored for later insertion.

[0064] In some embodiments, the user can define information to put intoportions 508, 708, 1504. This could include such things as stocktickers, stock alerts, weather information, traffic information,emergency alerts, news headlines, sports scores, movie theater showtimes, program guide listings, e-mail notifications, instant messagingnotifications, caller ID information, etc. The defined information couldinclude text, scrolling text, barker text, graphics, and/or video. Theuser could specify the portion 508, 708, 1504 to be used for a subset ofchannels or all channels. Presenting in the portion 508, 708, 1504 couldbe conditional on the channel and/or video format. For example, thepillar portions 708 could include weather information on channel four ifthe source aspect ratio was 4:3, etc.

[0065] In some embodiments, the headend 924 could distribute messagesfor display on a subset of the displays 932. The subsets could bedefined by program tier, subscriptions, geographic region, demographicinformation, personal preferences, address of the account holder, or anyother distinguishing factor. The messages could include still pictures,a graphic, text, scrolling text, barker text, audio, video in a window,and/or audio and/or video overlay. In one application, a emergencybroadcast is performed to a limited geographic region that includes asiren and a barker. In another application, a promotional message issent to a particular geographic region. The set top box or videoconverter 920 could determine a particular message is applicable to theuser and insert the message in an unused portion 508, 708, 1504, couldoverlay part of the screen or simply choose a portion 1504 to replacewith the message. The user could acknowledge and disable the distributedmessage.

[0066] Referring next to FIG. 17, a flow diagram of an embodiment of aprocess 1700 for processing a video image 200, 400 for showing on adisplay 932 having a different aspect ratio. The video converter 920receives the source image and formatting video ARC data in step 1704.The video ARC data may be embedded in the video stream or receivedseparately. Once playback is requested in step 1706, the video converterlocates the video ARC data and begins processing the video frames.

[0067] Each cutout or aperture 1308 is addressed in succession for theframe in steps 1708, 1712, 1216, and 1720 to formulate the portions ofthe displayed image 1508. In step 1708, the location of the cutout 1308is determined. Any specified scaling is performed in step 1712. Therotation and mirroring that might be specified is performed in step1716. In step 1720, the processed cutout is stitched into the specifiedportion 1504 of the image 1508. Where there are more cutouts 1308specified in the video ARC data, processing loops back to step 1708.Once all specified portions are added to the image 1508, it is displayedin step 1728.

[0068] A number of variations and modifications of the invention canalso be used. For example, some embodiments could allow the user tooverride the image processing to be performed automatically according tothe embedded video ARC data. Alternatively, the user could disable theautomatic processing in favor of a letterbox or pillar box presentation.Alternatively, users could influence the metadata according to theirpreference. Typically, the producer decides the various parameters formanipulation, but the user could take over to modify any of the videoARC parameters, for example, to move the 4:3 displayed image apertureright or left through the source 16:9 image 200.

[0069] The electronics that modify the image could be located in theuser location or away from the user location in various embodiments. Fora broadcast program, the video converter 920 could recognize themetadata and apply the image manipulation for the display 932. Forexample, a set up screen in the set top box could allow the user tospecify the aspect ratio preferred for the output of the set top box.Certain aspect ratios could be presumed. For example, the RCA andS-Video outputs could presume a 4:3 aspect ratio for the display and theHDTV could presume a 16:9 aspect ratio for the display. Both outputtypes could be active at the same time such that one produced aprocessed image and the other produced the unprocessed image.

[0070] For displays that allow single-cast programs such as some ondemand systems, the user could specify to the network node or headend,the aspect ratio of the display 932. The image processing could be donein the network node or headend before it is single casted to the userlocation. Systems like vDSL, that single cast all channels ofprogramming, could pre-process the image in this way also.

[0071] Although the above conversion discussion focused primarily on 4:3to 16:9 conversions, any aspect ratios could be performed in a similarmanner. For example, aspect ratios for various theatrical releases, PAL,NTSC, etc. could be converted. The video producer could specifyparameters for any number of display aspect ratios. The display aspectratios could even include the various computer display aspect ratios.

[0072] Some embodiments could have different metadata to allow remotemanipulation of the aspect ratio. That is to say, the metadata could bespecified in any number of ways such that the proper portion from thesource image is produced on the display. Any way to specify croppingand/or zooming could be used. In some embodiments, the metadata could beincluded with the video stream, while other embodiments could send theinformation to manipulate the image separately from the image. Indeed,the manipulation information could be sent through a different datachannel that that used for the image.

[0073] Although some of the above embodiments contemplate programdistribution to televisions, the invention can be applied to DVDs andtheaters. A DVD today may store multiple aspect ratios. In oneembodiment, the DVD could store a single aspect ratio and include ARCdata to instruct the DVD player or other equipment on doing theconversion. Similarly, the ARC data could instruct the movie theater onhow to accomplish any aspect ratio by adjusting the projection equipmentor screen.

[0074] Digital Broadcast Format Statistical Multiplexer

[0075] An embodiment of the present invention uses ahi-definition/hi-resolution statistical multiplexer (H/HSM) to format anumber of programs over multiple cable channels or multiple satellitetransponders. These transponders or cable channels are used by cable andsatellite television broadcasters to send a number of programssimultaneously over multiple carriers. In some circumstances, atransponder is used by a single content provider or shared among anumber of content providers. The programs are distinguished in thedatastream by program identifiers (PIDs). The PIDs are used to route thevarious programs in the datastream to different display channels. When auser selects a display channel, the corresponding program in thedatastream is decoded for display on the television.

[0076] Digital television formats are divisible into subsets, such as,high definition television (HDTV) and high resolution television (HRTV).Generally, HDTV has better quality than HRTV, but with better qualitycomes higher bandwidth requirements from the datastream. For example, anumber of HRTV programs could consume the same bandwidth of a singleHDTV program. As those skilled in the art can appreciate, conservationof bandwidth is desirable.

[0077] From the viewer's perspective, the quality of some programs issimilar irrespective of whether HDTV format or HRTV format is used. Forexample, many believe filmed programs are represented adequately inHRTV, but sports and concerts are better suited for HDTV. A givendisplay channel could at times provide programs in HDTV format andprovide programs in HRTV format at other times.

[0078] A display channel that runs movies and sporting events, forexample, might use 480 i HRTV for the films and 1080 i HDTV for thegames. If, for example, the display channel had a schedule thatconsisted of ⅔ movies and ⅓ films, it would transmit in the moreefficient mode ⅔ of the time. Because the 480 i HRTV transmissions useonly about ⅓ the bandwidth in some cases, this kind of a scenario wouldenable the cable or satellite broadcaster to add an entire HRTV programon average to the datastream versus what would be available if thenetwork telecast in 1080 i HDTV full time. More specifically, when thefirst display channel were in HDTV mode there is no extra bandwidth, butthere is bandwidth for two extra programs when in HRTV mode.

[0079] The H/HSM takes the programs and adds them to the datastream. Asdisplay channels switch between HDTV and HRTV formats, the H/HSMallocates and de-allocates bandwidth. Metadata could be sent with theprogram to indicate the decode format to the set top box. Alternatively,a separate control data channel could transport the formattinginformation for each program. Some embodiments could automaticallyrecognize the format from the program stream itself.

[0080] With reference to FIG. 18A, a block diagram of an embodiment of adirect broadcast satellite (DBS) system 1800-1 is shown. A headend 924receives video programs from any of a satellite downlink 916, a widearea network (WAN) 910, a terrestrial antenna 912, and a media reader922. The video programs are processed by the headend 924 and coupled toa satellite uplink 1850 and satellite 1854. A number of set top boxes1820 use their respective satellite downlink 1858 to receive the videoprograms. The set top boxes 1820 could be stand-alone units or integralwith another piece of video equipment, for example, a television.Control/data information is transported between the headend 924 and settop boxes using a POTS network 1870.

[0081] Referring next to FIG. 18B, a block diagram of an embodiment of acable system 1800-2 is shown. This embodiment communicatesbi-directionally between the headend 924 and set top boxes 1820 using atransmission system 908. A hybrid fiber/coaxial plant is used in thetransmission system 908. Other embodiments could distribute programsusing microwave transmission or VDSL, for example.

[0082] With reference to FIG. 19A, a block diagram of an embodiment of avideo processing system 1900-1 for the DBS system 1800-1. The headend924 processes video program streams 1932 from content providers.Generally, the video program streams 1932 are received in HDTV format,but could also be HRTV format. A video source processor 1904 determinesif each program stream 1932 should be broadcast in either HDTV or HRTVformat. This determination could be automatic and/or manual.

[0083] The video source processor 1904 could determine those programswhich would suffer least from down-conversion to HRTV format. Forexample, 24 frame per second movies could be detected and automaticallydown-converted to HRTV. The available bandwidth or other considerationscould be used when automatically down-converting some of the videoprogram streams 1932. The content provider could control down-conversionor at least state a “best-efforts” preference to the video sourceprocessor. This preference information could be metadata or transportedseparately from the video program stream 1932. In some cases, thecontent provider could preemptively send a HRTV video program stream tothe video source processor 1904 such that any down-conversion decisionis obviated. In one embodiment, the video source processor 1904down-converts film movies and some shows, but not sport events andconcerts.

[0084] A H/HSM 1908 includes ten multiplexers 1914. Each multiplexer1914 is coupled to a QAM-256 modulator 1920 and a satellite transponder1924. Other modulation and transponders could be used in otherembodiments. In this embodiment, each of the ten multiplexers 1914 cantransport two HDTV video streams, six HRTV video streams, or one HDTVvideo stream and three HRTV video streams. Other embodiments could usedifferent modulation rates, various HDTV and HRTV formats, differentmultiplexing ratios, different numbers of multiplexers or modulators,etc. Further, some embodiments could over-subscribe the HDTV multiplexerslots such that the video source processor 1904 is forced to do somedown-conversion to broadcast all the video streams 1932.

[0085] In some cases, all program streams remain in HDTV format andoccupy all twenty HDTV multiplexer slots. Most of the time, the videosource processor 1904 is reducing some of the twenty video streams 1932to HRTV such that all the multiplexer slots are not occupied. Fillerbandwidth is coupled to the video source processor for use in the unusedmultiplexer 1914 slots. The filler bandwidth could be occupied for anydigital information the set top box 1820 could receive, for example,advertisements, software, an additional program, VOD/SVOD programs forSTB storage, program guide information, promotional trailers, DOCSISdata, VOIP data, firmware upgrades, Internet data, interactivetelevision information, and/or other programs in the datastream havinghigher-bandwidth formats.

[0086] The set top boxes 1820 know which transponder and PIDs should bechosen to get a particular program for a particular display channel. Forexample, display channel three hundred and twelve could correspond tothe fourth HRTV slot on the eighth transponder 1924-8. As the videosource processor 1904 changes which video streams 1932 aredown-converted over time, the slot and transponder 1924 could change. Acontrol/data channel to the set top box 1820 is used to update themapping between display channel and transponder/slot. In this way, aparticular display channel corresponds to a particular video stream 1932even though the transport path and PIDs may change.

[0087] Referring next to FIG. 19B, a block diagram of another embodimentof the video processing system 1900-2 for the cable system 1800-2 isshown. In this embodiment, the modulators 1920 are combined onto asingle conductor or fiber by the RF combining network 1922 beforedistribution over the hybrid fiber/coaxial plant.

[0088] With reference to FIG. 20A, a block diagram of an embodiment of aset top box 1820-1 is shown. Satellite or cable program streams aredecoded in the program receiver 3012 and coupled to the displayinterface 3016. A control transceiver 2024 receives the mapping to thedisplay channel such that the controller 2014 can manipulate the programreceiver 2012 to produce the proper program. The display channel isdisplayed on the channel display 2020 and changed with commands receivedby the remote control receiver 2028.

[0089] Referring next to FIG. 20B, a block diagram of another embodimentof the set top box 1820-2 with program storage capability is shown. Thisembodiment stores programs in the program store 2008 to enable DVRand/or SVOD functionality. Further the filler data could be stored inthe program store 2010 for later use. For example, the filler data mayinclude infomercials that are stored in the program store 2008.

[0090] While the principles of the invention have been described abovein connection with specific apparatuses and methods, it is to be clearlyunderstood that this description is made only by way of example and notas limitation on the scope of the invention.

What is claimed is:
 1. A method for processing a video program having afirst aspect ratio different from a second aspect ratio of a display,the method comprising the steps of: specifying video conversioninformation for the video program at a first location, wherein the videoconversion information can be used to modify the video program from thefirst aspect ratio to the second aspect ratio; transmitting the videoprogram having the first aspect ratio and the video conversioninformation to a second location geographically away from the firstlocation; and processing the video program with the video conversioninformation at the second location to conform with the second aspectratio, whereby the video program can be viewed in the second aspectratio.
 2. The method for processing the video program having the firstaspect ratio different from the second aspect ratio of the display asrecited in claim 1, wherein the video conversion information dynamicallychanges throughout playback of the video program.
 3. The method forprocessing the video program having the first aspect ratio differentfrom the second aspect ratio of the display as recited in claim 1,wherein the specifying step comprises specifying video conversioninformation for a plurality of target aspect ratios.
 4. The method forprocessing the video program having the first aspect ratio differentfrom the second aspect ratio of the display as recited in claim 1,wherein the video conversion information specifies a portion of thevideo program for shrinking when conforming to the display.
 5. Themethod for processing the video program having the first aspect ratiodifferent from the second aspect ratio of the display as recited inclaim 1, wherein the video conversion information specifies a portion ofthe video program to display as the second aspect ratio.
 6. The methodfor processing the video program having the first aspect ratio differentfrom the second aspect ratio of the display as recited in claim 1,wherein the processing step comprises non-linearly scaling a portion ofthe video program to produce the second aspect ratio.
 7. The method forprocessing the video program having the first aspect ratio differentfrom the second aspect ratio of the display as recited in claim 1,wherein the processing step comprises rotating or mirroring a portion ofthe video program to produce the second aspect ratio.
 8. The method forprocessing the video program having the first aspect ratio differentfrom the second aspect ratio of the display as recited in claim 1,wherein the processing step comprises processing a plurality of discreetportions of the video program to produce the second aspect ratio.
 9. Acomputer-readable medium having computer-executable instructions forperforming the computer-implementable method for processing a videoprogram of claim
 1. 10. A video distribution system that distributes avideo program in a first aspect ratio and includes information toconvert the first aspect ratio to a second aspect ratio, the videodistribution system comprising: a distribution point; and a videoconverter remote the distribution point, wherein: the video converterreceives the information and the video program from the distributionpoint; and the video converter uses the information to guide conversionbetween the first aspect ratio and the second aspect ratio.
 11. Thevideo distribution system that distributes the video program in thefirst aspect ratio and includes information to convert the first aspectratio to the second aspect ratio as recited in claim 10, furthercomprising a video display coupled to the video converter, wherein thevideo display uses the second aspect ratio.
 12. The video distributionsystem that distributes the video program in the first aspect ratio andincludes information to convert the first aspect ratio to the secondaspect ratio as recited in claim 10, wherein the information guidesconversion between the first aspect ratio and a third aspect ratio. 13.The video distribution system that distributes the video program in thefirst aspect ratio and includes information to convert the first aspectratio to the second aspect ratio as recited in claim 10, wherein theinformation guides conversions that change at least once during thevideo program.
 14. The video distribution system that distributes thevideo program in the first aspect ratio and includes information toconvert the first aspect ratio to the second aspect ratio as recited inclaim 10, wherein: the information identifies a first cutout and asecond cutout of a frame of the video program; and the informationindicates how the first and second cutouts are used in the second aspectratio.
 15. A computer data signal embodied in a carrier wave, thecomputer data signal comprising: first information that represents avideo program in a first aspect ratio; and second information thatindicates how the first aspect ratio is converted to a second aspectratio, wherein: the computer data signal passes between a distributionpoint and a video converter; and the distribution point is remotelylocated from the video converter.
 16. The computer data signal embodiedin a carrier wave as recited in claim 15, wherein the carrier wave ispassed in at least one of a following manners: wired, wirelessly,optically, terrestrially, and via satellite.
 17. The computer datasignal embodied in a carrier wave as recited in claim 15, wherein thesecond information indicates a first manipulation at a first temporalpoint in the video and a second manipulation at a second temporal pointin the video.
 18. The computer data signal embodied in a carrier wave asrecited in claim 15, further comprising third information that indicateshow the first aspect ratio is converted to a third aspect ratio.
 19. Thecomputer data signal embodied in a carrier wave as recited in claim 15,wherein a plurality of discreet portions of the video program are usedto produce the second aspect ratio.